Dynamic search input selection

ABSTRACT

Described is a system and method for enabling dynamic selection of a search input. For example, rather than having a static search input box, the search input may be dynamically positioned such that it encompasses a portion of displayed information. An image segment that includes a representation of the encompassed portion of the displayed information is generated and processed to determine an object represented in the portion of the displayed information. Additional images with visually similar representations of objects are then determined and presented to the user.

BACKGROUND

With the ever expanding amount of accessible digital content availableto users and customers, it continues to become more and more difficultto discover the content for which the user is searching. Severaldifferent search techniques exist, such as keyword searching, but thereare many inefficiencies in such systems.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures.

FIGS. 1-6 are representations of a graphical user interface forselecting an object of interest using a dynamic search control,according to an implementation.

FIGS. 7-8 are representations of another graphical user interface forselecting an object of interest using a dynamic search control,according to an implementation.

FIG. 9 is a flow diagram illustrating an example dynamic search process,according to an implementation.

FIG. 10 is a flow diagram illustrating another example dynamic searchprocess, according to an implementation.

FIG. 11 is a flow diagram illustrating an example image searchprocessing process, according to an implementation.

FIG. 12 is a flow diagram illustrating an example keyword determinationprocess, according to an implementation.

FIG. 13 is a flow diagram illustrating an example search resultrefinement and selection process.

FIG. 14 illustrates an example computing device, according to animplementation.

FIG. 15 illustrates an example configuration of components of acomputing device, such as that illustrated in FIG. 14.

FIG. 16 is a pictorial diagram of an illustrative implementation of aserver system that may be used for various implementations.

While implementations are described herein by way of example, thoseskilled in the art will recognize that the implementations are notlimited to the examples or drawings described. It should be understoodthat the drawings and detailed description thereto are not intended tolimit implementations to the particular form disclosed but, on thecontrary, the intention is to cover all modifications, equivalents andalternatives falling within the spirit and scope as defined by theappended claims. The headings used herein are for organizationalpurposes only and are not meant to be used to limit the scope of thedescription or the claims. As used throughout this application, the word“may” is used in a permissive sense (i.e., meaning having the potentialto), rather than the mandatory sense (i.e., meaning must). Similarly,the words “include,” “including,” and “includes” mean including, but notlimited to.

DETAILED DESCRIPTION

Described herein are systems and methods that facilitate searching forinformation based on a selection of a portion of a larger image. Ratherthan attempting to process an entire image and/or requiring a searchinput (image, keyword(s), etc.) be provided at a statically positionedsearch control, the described systems and methods provide a dynamicallypositionable search control that may be positioned over any portion ofdisplayed information. For example, the search control may be visuallypresented such that it appears to overlay the displayed informationpresented on a user interface and encompasses a portion of the displayedinformation. A user may move or alter the position of the search controland/or adjust the size of the search control. As the user moves thesearch control and/or alters the size of the search control, additionalimages similar to an object encompassed by the search control aredetermined and presented to the user. As the user continues to moveand/or alter the size of the search control, the additional images areupdated so that they correspond with the object represented in theportion of the displayed information that is encompassed by the searchcontrol.

When the search control is at a desired position (e.g., the user stopsmoving and/or altering the size of the search control), in addition topresenting additional images that are visually similar and/or include arepresentation of a visually similar object to the representation of theobject encompassed by the search control, one or more keywords may bepresented to the user. The keywords may be selected to alter whichadditional images are determined for presentation to the user.

A user may interact with the keywords and/or the presented additionalimages. For example, a user may select one or more of the keywords andthe presentation of the additional images will be updated to presentadditional images that correspond to the keyword and that are visuallysimilar to the portion of the displayed information encompassed by thesearch control. Likewise, the user may interact with the presentedadditional images, such as by viewing, selecting, etc., the additionalimages. When an additional image is selected, the user may be presentedwith one or more actions associated with the selected additional imagethat may be initiated. Example actions include, but are not limited to,initiating a purchase of an object represented in the image, sharing theimage, saving the image, emailing the image, obtaining additionalinformation about the image and/or the object represented in the image,etc.

By providing a dynamically positionable search control, a user canselect a portion of displayed information that is of interest, or thatincludes a representation of an object of interest, and receive searchresults that are relevant to that portion of the displayed informationand/or object of interest, rather than having to search an entire imageand/or receive results that may or may not be of interest to the user.Likewise, by updating the presented additional images as the user movesthe search control, the user can explore, in a systematic fashion, thedisplayed information and view on an object-by-object basis additionalimages that include representations of objects that are visually similarto those included in the displayed information.

FIGS. 1-6 are representations of user interfaces for dynamicallypositioning a search control and selecting an object of interest,according to an implementation. A user may be presented with a graphicaluser interface via a client device that includes displayed information.A client device may include, for example, personal computers, tabletcomputers, eBook reader devices, laptop computers, desktop computers,netbooks, personal digital assistants (PDA), portable gaming devices,wireless communication devices, such as smart phones or mobile handsets,set-top boxes, game consoles, cameras, or any combination thereof.Utilizing a client device, a user may interact with the displayedinformation.

Turning first to FIG. 1, illustrated is a user interface 100 thatincludes displayed information, such as an image of Tiger Leggings 102,an image of a man in a suit 104, a portion of an image 106, and aportion of an image 108. As will be appreciated, the displayedinformation may be any type, style, quantity, etc., of information thatcan be visually presented to a user. For example, displayed informationmay include images, videos, text, web pages, email, word documents,presentations, spreadsheets, etc., or any combination thereof.

When a user is searching for information and/or items, they are oftenonly interested in specific objects or portions of the displayedinformation represented on the user interface 100. For example, a usermay only be interested in leggings 102-1, the shoes 102-2, the man'snecktie 104-1, the man's suit jacket 104-2, etc. To obtain additionalinformation and/or additional images that include visually similarobjects to those of interest, a user may initiate or invoke the dynamicsearch control by selecting the search control icon 110.

Upon selecting the search control icon 110, the search control isvisually presented on the user interface 100. For example, FIG. 2illustrates a graphical user interface 200, which corresponds to theuser interface 100, with the dynamic search control 205 activated. Asillustrated, the dynamic search control 205 is visually presentedconcurrently with the displayed information such that the dynamic searchcontrol visually appears to be presented over the displayed informationand encompassing less than all of the displayed information.

A user may interact with the dynamic search control by moving it aroundon the user interface 200, adjusting the size of the search control,etc. As the user adjusts the search control (e.g., moves and/or altersthe size of the search control), additional images 210 are determinedand presented to the user. As illustrated in FIG. 2, the user haspositioned the search control over the image 204 of the man in the suitand adjusted the size of the search control 205 so that it primarilyencompasses the necktie 204-1 represented in the image 204. Thedisplayed information that is not encompassed by the dynamic searchcontrol 205, such as the image 202 of the leggings, may be obfuscated orgreyed out, or otherwise distinguished from the search control 205.

The dynamic search control may be of any size or shape and may bedynamically positioned at any location on the user interface 200. Inthis example, the dynamic search control 205 is substantially circularand the radius of the substantially circular shape may be increased ordecreased using the size controls 206. In other implementations, thedynamic search control may be of any size and/or shape. For example, thedynamic search control 205 may be square, rectangular, octagonal,elliptical, or any other polygonal shape or other shape that encompassesa portion of the displayed information.

A user may adjust the size and/or position of the search control 205 byinteracting with the search control. For example, if the client deviceupon which the displayed information is presented includes a touchsensitive display, a user may alter the position and/or size of thesearch control by touching or otherwise interacting with the touchsensitive display and moving the search control to a desired position onthe user interface and/or adjusting the search control to a desiredsize. Other forms of adjustment may likewise be utilized, including butnot limited to voice control, mouse, keyboard, etc.

In this example, the user has adjusted the size of the search control205 and positioned the search control 205 such that it encompasses aportion of the image 204 that includes the necktie 204-1. Asillustrated, the portion of the displayed information presented on theuser interface 200 that is encompassed by the search control 205 isdistinguished from the other portions of the displayed information.

As the user moves the search control over different portions of thedisplayed information, an image segment that includes the encompassedportion of the displayed information is generated. As discussed furtherbelow with respect to FIGS. 9 and 10, in some implementations, an imageof the entire displayed information may be obtained and cached on theclient device. As the search control is moved, the position of thesearch control is utilized to obtain corresponding image segments fromthe cached image of the displayed information. In other implementations,the page information (e.g., uniform resource locator, image signature)and the search control position information may be sent to a remotecomputing resource and the remote computing resource may utilize thatinformation to generate an image segment. The image segment includes theportion of the displayed information that is encompassed by the searchcontrol.

Image segments may be periodically generated as the search control ismoved and/or otherwise adjusted. In some implementations, image segmentsmay be generated at defined intervals (e.g., every second, every twoseconds) as the search control is being moved or adjusted. In otherimplementations, a frequency of image segment generation may be based ona speed and/or direction in which the search control is moving, based ona size of the search control, based on color variations in the displayedinformation, based on user settings or preferences, etc. In otherimplementations, an image segment may not be obtained until it isdetermined that the search control 205 is at a desired position and/orsize. For example, it may be determined that the search control is at adesired position by, for example, determining that a defined amount oftime (e.g., 2 seconds) has elapsed without any alteration of theposition and/or size of the search control 205, in response to receivingan indication from a user (e.g., the user releasing a mouse button orremoving a touch input from a touch sensitive screen), etc.

As each image segment is generated, the image segment is processed todetermine an object represented in the image segment, referred to hereinas an object of interest, and other additional images that includevisually similar objects, and/or that are visually similar to the imagesegment, are determined Processing the image segment and determiningadditional, visually similar images is discussed in further detail belowwith respect to FIG. 11.

The visually similar images are presented to the user as additionalimages 210. In this example, a portion of the image segment 208corresponding to the displayed information that is encompassed by thesearch control 205 is presented concurrently with the additional images210 as part of the user interface 200. Any number of additional images210 may be presented. In this example, there are seven additional images210-1, 210-2, 210-3-210-N presented on the user interface 200 that aredetermined to be visually similar to the object represented in theportion of the displayed information that is encompassed by the searchcontrol 205. In other implementations, there may be fewer or moreadditional images presented on the user interface 200.

Additional images that are determined to include a representation of anobject that is visually similar to the selected object of interest may,in some implementations, be ranked and those having the highest rank(e.g., based on visual similarity to the selected object of interest)are presented as the additional images 210.

As the user moves or otherwise alters the search control 205, additionalimage segments are generated and processed and additional imagescorresponding to the representation of the object encompassed by thesearch control are presented to the user. For example, FIG. 3illustrates a user interface 300 in which the user has altered the sizeand position of the search control 305 from the representation of thenecktie, as illustrated in FIG. 2, such that the search control 305encompasses the representation of the suit jacket 304-2. Specifically,in this example, the user has increased the radius of the search control305 by manipulating the size controls 306 and adjusted the position ofthe search control 305. An image segment is generated that includes theportion of the displayed information that is encompassed by the searchcontrol 305. The image segment is then processed and it is determinedthat the object of interest represented in the image segment is themen's suit jacket. In this example, even though there are multiplerepresentations of objects encompassed by the search control, namely therepresentation of the suit jacket 304-2 and the representation of thenecktie 304-1, the processing may determine that the object of interestis the suit jacket. This determination may be based on the size and/orposition of the search control, user preferences, and/or otherinformation. In other implementations, each object represented in theimage segment may be determined.

As discussed above, based on the object represented in the imagesegment, additional images 310 with visually similar representations ofobjects are determined and presented to the user. Likewise, arepresentation of the image segment 308 may also be presented to theuser concurrently with the additional images.

Continuing with the example, as the user continues to alter the positionand/or size of the search control, additional image segments aregenerated and processed and additional images corresponding to therepresented object that is encompassed by the search control aredetermined and presented to the user. Updating the additional images asthe user alters the position and/or size of the search control providesa real-time exploration of the displayed information and providesadditional images that are relevant to the portion of the displayedinformation that is of interest to the user and encompassed by thesearch control.

For example, turning to FIG. 4, the user has altered the position of thesearch control 405 such that it is positioned over the representation ofthe tiger leggings 402-1 included in the image 402 of the displayedinformation that is presented on the user interface 400. As discussedabove, an image segment is generated that includes the portion of thedisplayed information encompassed by the search control 405 and thatimage segment is processed to determine the additional images 410 thatare presented to the user as part of the user interface 400.

In some implementations, in addition to presenting additional images410, keywords 412 that correspond to the additional images may bepresented. Like the additional images 410, any number of keywords 412may be presented to the user for selection. As discussed further belowwith respect to FIG. 12, keywords may be associated with one or more ofthe additional images 410 and may be ranked based on the frequency withwhich the keywords are associated with the additional images. Keywords412 with the highest frequency may be presented on the user interface.In this example, there are ten keywords 412-1, 412-2, 412-3-412-N thatare presented on the user interface. In other implementations, there maybe fewer or additional keywords presented on the user interface 400.

In some implementations, the keywords may only be presented when thesearch control has been stationary for a determined period of time(e.g., 2 seconds). In other implementations, the keywords may bepresented at the same frequency as the additional images.

A user may select an additional image 410, select one or more of thekeywords 412 and/or reposition or resize the search control 405. If theuser repositions and/or resizes the search control 405, a new imagesegment is generated and additional images and/or keywords correspondingto that new image segment are determined and presented to the user. Ifthe user selects one or more of the keywords 412, the order in which theadditional images are presented is adjusted based on the selectedkeyword(s) and the similarity of the additional images with the imagesegment. Likewise, the order in which the keywords are presented may beupdated based on the additional images that correspond to the selectedkeyword.

For example, FIG. 5 illustrates a user interface 500 in which thepresentation of the order of the additional images 510 and the order ofthe keywords 512 is updated in response to selection of the two keywords512-1, 512-2 of “Leggings” and “Print Leggings.” As discussed furtherbelow with respect to FIG. 13, each time a keyword is selected, theadditional images that are associated with the selected keywords aredetermined and re-ranked. In this example, the additional images inwhich both selected keywords 512-1, 512-2 are associated with theadditional image are determined and ranked higher than other additionalimages. For the additional images that include both keywords, they arefurther ranked based on the similarity with the image segment. Forexample, the first two additional images 510-1 and 510-2 are eachassociated with both keywords 512-1, 512-2, and as such are rankedhigher than the other additional images 510-3-510-N. Likewise,additional image 510-1 has a higher similarity to the image segment andis ranked higher than additional image 510-2. A user may continueselecting and/or un-selecting keywords and the presented order of theadditional images 510 will be updated accordingly.

In some implementations, only additional images with selected keywordassociations may be presented. In other implementations, as illustratedin FIG. 5, the additional images with the associated keywords may beranked higher than other additional images that do not have the selectedkeywords associated therewith.

Based on the selected keywords and the additional images determined tobe associated with the selected keywords, the order in which thekeywords are presented may be adjusted. For example, as discussedfurther below with respect to FIG. 11, upon selection of a keyword, theadditional images associated with the selected keyword are determinedLikewise, other keywords associated with those additional images may bedetermined and the keywords re-ranked so that keywords associated withthe additional images that are associated with the already selectedkeyword may be ranked higher than keywords associated with additionalimages that do not have the already selected keyword associatedtherewith. The re-ranked keywords may then be presented to the userconcurrently with the re-ranked additional images.

A user may also select an additional image 510 and obtain additionaldetails and/or actions with respect to the selected additional image.For example, FIG. 6 includes a user interface 600 that is presented inresponse to a user selecting the additional image 510-3 (FIG. 5). Asillustrated, the selected additional image is presented in a larger viewas additional image 610-3. Likewise, one or more actions 602 that may beinitiated with respect to the selected additional image are alsopresented. The additional actions include, but are not limited to,“Save” 602-1, “Share” 602-2, and “Buy” 602-N or purchase. In otherimplementations, there may be additional, fewer and/or differentactions. Other actions that may be associated or initiated with respectto a selected additional image include, but are not limited to, a sendaction (e.g., email the additional image), a post action (e.g., post theadditional image to a social website), etc.

A user may select one or more of the actions and the correspondingaction is initiated with respect to the additional image. For example,if the user selects the “Buy” 602-N action, the action of purchasing theobject represented in the additional image, or purchase of theadditional image, is initiated. Initiation of the action may includeobtaining additional details regarding the represented object, providingprice information, seller information, shipping details, etc. As will beappreciated, any variety of actions may be associated with one or moreof the additional images and those actions may be initiated with respectto a selected additional image.

FIG. 7 illustrates a user interface 700 presented on another clientdevice, according to an implementation. This example user interface 700corresponds to the user interface 500 (FIG. 5) and is presented on adifferent form factor client device 701. As with the above example, theuser has positioned the search control 705 and adjusted the size of thesearch control using the size controls 706 such that the search control705 is positioned to encompass the representation of the tiger leggings702-1 presented in the image 702 that is included in the displayedinformation presented on the user interface 700 by the client device701. Likewise, the user has selected the two keywords 712 “Leggings” and“Print Leggings.” As discussed above, an image segment is generated thatincludes the portion of the displayed information that is encompassed bythe search control 705, determined additional images 710, re-rankedthose additional images 710 based on the selected keywords 712 andpresented the re-ranked additional images 710 on the user interface 700of the client device 701. In this example, additional image 710-1corresponds to additional image 510-1, additional image 710-2corresponds to additional image 510-2, additional image 710-3corresponds to additional image 510-3, additional image 710-Ncorresponds to additional image 510-N, etc.

Regardless of the user experience or the client device used, theimplementations described herein enable a user to dynamically selectportions of displayed information and obtain additional images that arevisually similar to objects represented in the selected portion of thedisplayed information. The user may interact with those additionalimages, select presented keywords, continue to explore the displayedinformation by continuing to move the search control, etc. For example,referring to FIG. 8, like the above example, the user has selected theadditional image 710-3 and is presented with additional image 810-3along with actions 802-1, 802-2-802-N that may be initiated by the user.

FIG. 9 is a flow diagram illustrating an example dynamic search process900, according to an implementation. This process, and each processdescribed herein, may be implemented by the architectures describedherein or by other architectures. The process is illustrated as acollection of blocks in a logical flow graph. Some of the blocksrepresent operations that can be implemented in hardware, software, or acombination thereof. In the context of software, the blocks representcomputer-executable instructions stored on one or more computer readablemedia that, when executed by one or more processors, perform the recitedoperations. Generally, computer-executable instructions includeprocesses, programs, objects, components, data structures, and the likethat perform particular functions or implement particular abstract datatypes.

The computer readable media may include non-transitory computer readablestorage media, which may include hard drives, floppy diskettes, opticaldisks, CD-ROMs, DVDs, read-only memories (ROMs), random access memories(RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards,solid-state memory devices, or other types of storage media suitable forstoring electronic instructions. In addition, in some implementations,the computer readable media may include a transitory computer readablesignal (in compressed or uncompressed form). Examples of computerreadable signals, whether modulated using a carrier or not, include, butare not limited to, signals that a computer system hosting or running acomputer program can be configured to access, including signalsdownloaded through the Internet or other networks. Finally, the order inwhich the operations are described is not intended to be construed as alimitation, and any number of the described operations can be combinedin any order and/or in parallel to implement the process.

The example process 900 begins upon receipt of an invocation or requestto initiate a search using the dynamically adjustable search control, asin 902. Upon receiving an invocation, the dynamic search control ispresented on a user interface such that it appears to be positioned overthe displayed information that is presented on the user interface, as in904. For example, a portion of the displayed information that is notencompassed by the search control may be obfuscated, greyed out, hiddenfrom view, etc., such that the dynamic search control and the portion ofthe displayed information that is encompassed by the search control aredistinguished. In other implementations, the portion of the displayedinformation that is encompassed by the search control may behighlighted, magnified, or otherwise distinguished from other portionsof the displayed information so that a user can discern which portion ofthe displayed information is encompassed by the search control.

A user may interact with the search control by, for example,repositioning the search control to a different position on the userinterface such that different portions of the displayed information areencompassed by the search control. Likewise, the size and/or shape ofthe search control may be altered such that more or less of thedisplayed information is encompassed by the search control.

As the search control is moved and/or the size adjusted, positioninformation corresponding to the search control is determined, as in908. The position information includes the size, shape and/or dimensionsof the search control and the horizontal and vertical position of thesearch control on the user interface and/or with respect to thedisplayed information. For example, a center point of the search controlmay be determined, a horizontal and vertical position of the centerpoint on the user interface determined, and dimensions (e.g., height,width, shape) of the search control determined with respect to thecenter point. In another example, the perimeter or perimeter points ofthe search control may be determined in horizontal and verticalpositions with respect to the user interface.

The position information of the search control and the user interfaceinformation corresponding to the displayed information presented on theuser interface are sent to a remote computing resource, as in 910. Theuser interface information may include, for example, a uniform resourcelocator (URL) of a web page that is presented as the displayedinformation on the user interface. As another example, the userinterface information may include an image signature corresponding to animage within the displayed information that is partially encompassed bythe search control. In such implementations, the remote computingresource maintains and/or has provided the displayed information that ispresented on the client device. For example, the displayed informationmay correspond to a web page that has been sent or provided to a clientdevice by the remote computing resource. Alternatively, the displayedinformation may have been provided by another device to the clientdevice and to the remote computing resource. Regardless of the source,in the implementation discussed with respect to FIG. 9, the remotecomputing resource has information regarding the displayed informationthat is presented on the display of the client device.

The remote computing resource, upon receiving the user interfaceinformation and the position information of the search control,determines an image segment that corresponds to the portion of thedisplayed information encompassed by the search control, as in 912. Forexample, the remote computing resource may determine the displayedinformation and the position of the displayed information on the userinterface based on the received user interface information. Likewise,the portion of the displayed information that is encompassed within thesearch control may be determined by overlaying the position informationof the search control on the determined displayed information and theportion of the displayed information that is encompassed within theposition information may be segmented out as an image segment.

The image segment is then processed to determine an object representedin the image segment and a plurality of additional images that includerepresentations of objects that are visually similar to the objectrepresented in the image segment, as in 1100. Processing of the imagesegment is discussed in further detail below with respect to FIG. 11.Likewise, one or more keywords that are associated with the additionalimages may also be determined, as in 1200. Determination of keywords isdiscussed in further detail below with respect to FIG. 12.

The ranked additional images that are returned by the example process1100 and the ranked keywords that are returned by the example process1200 are sent by the remote computing resource to the client device, asin 914. In some implementations, all of the additional images and/oradditional keywords that are returned by the example process 1100 andthe example process 1200 may be sent to the client device. In otherimplementations, only a portion of the additional images and/oradditional keywords may be sent to the client device and the remotecomputing resource may maintain a record of all of the additional imagesand/or all of the additional keywords and an indication of whichadditional images and/or additional keywords have been sent to theclient device.

Finally, the client device, upon receiving the additional images and/oradditional keywords, presents the additional images, as in 916. Inaddition to presenting the additional images, a determination is made asto whether the dynamic search control has been repositioned, as in 918.If it is determined that the dynamic search control has beenrepositioned, the example process 900 returns to block 908 andcontinues, thereby updating the additional images that are presented tothe user to correspond to the representation of the object encompassedby the repositioned search control. If it is determined that the dynamicsearch control has not been repositioned, the ranked keywords arepresented to the user, as in 920. In some implementations, there may bea delay before it is determined that the search control has not beenrepositioned and before the keywords are presented. For example, theexample process may include a two second delay before it is determinedthat the search control has not been repositioned and the keywords arepresented. In some implementations, the image segment, the searchcontrol, and/or the displayed information may also be presented alongwith the additional images and/or additional keywords.

FIG. 10 is a flow diagram illustrating another example dynamic searchprocess 1000, according to an implementation. The example process 1000begins upon receipt of an invocation or request to initiate a searchusing the dynamically adjustable search control, as in 1002. Uponreceiving an invocation, the dynamic search control is presented on auser interface such that it appears to be positioned over the displayedinformation that is presented on the user interface, as in 1004. Forexample, a portion of the displayed information that is not encompassedby the search control may be obfuscated, greyed out, hidden from view,etc., such that the dynamic search control and the portion of thedisplayed information that is encompassed by the search control aredistinguished. In other implementations, the portion of the displayedinformation that is encompassed by the search control may behighlighted, magnified, or otherwise distinguished from other portionsof the displayed information so that a user can discern which portion ofthe displayed information is encompassed by the search control.

A user may interact with the search control by, for example,repositioning the search control to a different position on the userinterface such that different portions of the displayed information areencompassed by the search control. Likewise, the size and/or shape ofthe search control may be altered such that more or less of thedisplayed information is encompassed by the search control.

In addition to presenting the search control, an image of the displayedinformation that is presented on the user interface is obtained andcached or otherwise stored in a memory on the client device, as in 1007.In some implementations, an image of the displayed information isobtained and cached when the search control invocation is received(block 1002). In other implementations, the image of the displayedinformation is obtained and cached each time an image segment is to begenerated as the search control is moved by a user.

In addition to obtaining an image of the displayed information, positioninformation corresponding to the search control is determined, as in1008. The position information includes the size, shape and/ordimensions of the search control and the horizontal and verticalposition of the search control on the user interface and/or with respectto the displayed information. For example, a center point of the searchcontrol may be determined, a horizontal and vertical position of thecenter point on the user interface determined, and dimensions (e.g.,height, width, shape) of the search control determined with respect tothe center point. In another example, the perimeter or perimeter pointsof the search control may be determined in horizontal and verticalpositions with respect to the user interface.

Based on the position information, the obtained image of the displayedinformation is segmented, as in 1009. Specifically, an image segmentthat includes the portion of the obtained image corresponding to theencompassed displayed information is generated from the obtained andcached image of the displayed information. The image segment is thensent to the remote computing resource, as in 1010. In someimplementations, rather than generating the image segment on the clientdevice, the entire obtained image of the displayed information may besent to the remote computing resource and the remote computing resourcemay generate the image segments. In such an implementation, each timeposition information of the search control is determined, the positioninformation is sent to the remote computing resource and the remotecomputing resource uses the received position information and theobtained image of the displayed information to generate an imagesegment.

The remote computing resource processes the image segment to determinean object represented in the image segment and a plurality of additionalimages that include representations of objects that are visually similarto the object represented in the image segment, as in 1100. Processingof the image segment is discussed in further detail below with respectto FIG. 11. Likewise, one or more keywords that are associated with theadditional images may also be determined, as in 1200. Determination ofkeywords is discussed in further detail below with respect to FIG. 12.

The ranked additional images that are returned by the example process1100 and the ranked keywords that are returned by the example process1200 are sent by the remote computing resource to the client device, asin 1014. In some implementations, all of the additional images and/oradditional keywords that are returned by the example process 1100 andthe example process 1200 may be sent to the client device. In otherimplementations, only a portion of the additional images and/oradditional keywords may be sent to the client device and the remotecomputing resource may maintain a record of all of the additional imagesand/or all of the additional keywords and an indication of whichadditional images and/or additional keywords have been sent to theclient device.

Finally, the client device, upon receiving the additional images and/oradditional keywords, presents the additional images, as in 1016. Inaddition to presenting the additional images, a determination is made asto whether the dynamic search control has been repositioned, as in 918.If it is determined that the dynamic search control has beenrepositioned, the example process 900 returns to block 908 andcontinues, thereby updating the additional images that are presented tothe user to correspond to the representation of the object encompassedby the repositioned search control. If it is determined that the dynamicsearch control has not been repositioned, the ranked keywords arepresented to the user, as in 920. In some implementations, there may bea delay before it is determined that the search control has not beenrepositioned and before the keywords are presented. For example, theexample process may include a two second delay before it is determinedthat the search control has not been repositioned and the keywords arepresented. In some implementations, the image segment, the searchcontrol, and/or the displayed information may also be presented alongwith the additional images and/or additional keywords.

FIG. 11 is a flow diagram illustrating an example image processingprocess 1100, according to an implementation. The example process 1100begins upon receipt of an image segment, as in 1102. As discussed above,an image segment is determined to correspond to a portion of displayedinformation of a user interface that is encompassed by a dynamicallypositionable search control.

The received image segment is processed to determine an objectrepresented in the image segment, as in 1104. One or more techniques maybe utilized to determine the object represented in the image segment.For example, one or more of an object detection algorithm, edgedetection algorithm, etc., may be used to detect the object. In otherimplementations, a color histogram or color signature corresponding tothe object represented in the image and/or the image segment may bedetermined and utilized as representative of the object represented inthe image. In some implementations, techniques described in commonlyowned U.S. patent application Ser. No. 14/279,871, entitled “Image BasedSearch,” the contents of which are incorporated by reference herein intheir entirety, may be used to determine the object represented in theimage segment.

Based on the determined object, one or more additional images containingvisually similar objects may be determined, as in 1106. For example,images maintained in a data store by the remote computing resource maybe processed using similar object detection techniques and indexed byobject type, shape, size, color, brand, etc. When the object representedin the image segment is determined, it may be compared with the indexmaintained by the remote computing resource to determine additionalimages that include objects having the same or similar type, shape,size, color, brand, etc. Additional images with objects determined to bevisually similar may be ranked or weighted based on how visually similarthe representation of the object in the image segment is to the objectrepresented in the additional image. For example, a single featurevector of the object represented in the image segment may be compared tofeature vectors of the visually similar objects to determine asimilarity between the objects.

Based on the similarity scores of the additional images, the additionalimages are ranked, as in 1108, and the highest ranked images arereturned, as in 1110. In some implementations, only a portion of animage with a visually similar object may be returned as an additionalimage. For example, only the segment of the additional image thatincludes the visually similar object may be returned. In otherimplementations, the entire additional image containing the visuallysimilar object may be returned. Likewise, the additional images may bereturned in an ordered set corresponding to the ranking and/or anadditional images ranking or additional images ranking list may beprovided with the additional images.

As discussed above, in some implementations, one or more keywords mayalso be determined that are also presented to the user. In suchimplementations, the example process 1200 (FIG. 12) may be performed todetermine the additional keywords. The example process 1200, illustratedin FIG. 12, begins by determining each of the keywords associated withthe additional images determined by the example process 1100 (FIG. 11),as in 1202. For example, some or all of the additional images may haveone or more keywords, attributes, metadata, descriptions, etc., that maybe analyzed and used as keywords. For each determined keyword, afrequency or count is determined based on how many of the additionalimages are associated with each keyword. For example, if the keyword“Leggings” is associated with 115 out of 900 additional images, thekeyword “Leggings” will have a frequency or count representative of the115 times it is associated with the 900 additional images.

The keywords are then ranked based on the determined frequency for eachkeyword, as in 1204, and the ranked keywords are returned, as in 1206.In some implementations, all of the additional keywords may be returnedin a ranked order. In other implementations, only a portion of theranked keywords may be returned. Likewise, the keywords may be returnedin an ordered set corresponding to the ranking and/or returned with akeyword ranking or keyword ranking list for the keywords. In someimplementations, the keyword ranking list may identify each of thekeywords along with a frequency of use of each keyword.

FIG. 13 is a flow diagram illustrating an example search resultrefinement and selection process 1300, according to an implementation.The example process 1300 begins by determining if a presented keywordhas been selected, as in 1302. If it is determined that a keyword hasbeen selected, the determined additional images are re-ranked based onthe selected keyword and the similarity of the additional images withthe image segment, as in 1304. For example, additional images with theselected keyword(s) associated therewith are given a higher weight thanother additional images such that they are ranked higher than the otheradditional images. Likewise, for additional images that include anassociation with the selected keyword(s), those additional images areranked according to their similarity with the image segment, such thatthe additional image with the associated keyword and the highest visualsimilarity will be ranked highest.

In addition to re-ranking the additional images based on a selectedkeyword, the keywords may also be re-ranked, as in 1305. For example,the set of keywords may be reduced to only include keywords associatedwith additional images that are associated with the selected keyword. Insuch an example, the reduced set of keywords may then be re-ranked basedon the frequency in which those keywords are associated with theadditional images. In one example, the frequency of the reduced set ofkeywords may be determined based on all of the additional images. Inanother example, the frequency of the reduced set of keywords may bedetermined based only on the additional images that are associated withthe selected keyword. The re-ranked images and/or re-ranked non-selectedkeywords are then sent and presented to the user, as in 1306.

In some implementations, the process of re-ranking the additional imagesand/or re-ranking the keywords may be performed on the client device. Inother implementations, the re-ranking of the additional images and/orthe keywords may be performed at the remote computing resource. Forexample, if the client device has received all of the additional imagesand corresponding keywords, upon selection of a keyword, the clientdevice may re-rank the additional images for presentation based on theselected keyword. Likewise, the client device may reduce the set ofkeywords and re-rank the reduced set of keywords. Alternatively, uponselection of a keyword, the client device may send an indication of theselected keyword to the remote computing resource. The remote computingresource may then re-rank the additional images based on the selectedkeyword, reduce the set of keywords and re-rank the reduced set ofkeywords and provide the client device with the re-ranked additionalimages and the re-ranked unselected keywords. Alternatively, the remotecomputing resource may provide the client device an updated additionalimage ranking list and/or an updated keyword ranking list indicating anorder in which the additional images and/or non-selected keywords are tobe presented, or a combination thereof. For example, if the remotecomputing resource only sent the client device the highest ranked imagesthat were presented, upon re-ranking, the remote computing resource maydetermine which additional images are to be presented, determine whichof those additional images have already been sent to the client device,and provide the client device with an updated additional images rankinglist and any additional images that are to be presented according to theupdated additional images ranking list. The client device, uponreceiving the updated additional images ranking list and any additionalimages that it did not already have, re-orders and presents theadditional images according to the updated additional images rankinglist. In a similar manner, the presentation of the keywords may likewisebe updated.

Returning to FIG. 13, a determination is then made as to whether anadditional keyword has been selected, as in 1308. If it is determinedthat an additional keyword has been selected, the example process 1300returns to block 1304 and continues, considering each selected keywordand the visual similarities between the additional images and the imagesegment.

If it is determined that no additional keywords have been selected atdecision block 1308, or if it is determined at decision block 1302 thatno keyword has been selected, a determination is made as to whether anadditional image is selected, as in 1310. If it is determined that anadditional image has not been selected, the example process 1300 returnsto decision block 1308 and continues. If it is determined that anadditional image has been selected, the selected additional image andone or more corresponding actions are presented, as in 1312. Forexample, the selected additional image may be presented in a larger sizeand/or with additional information about the selected additional image.Likewise, the one or more corresponding actions may be presented ascontrols that may be selected or initiated by the user. Example actionsinclude, but are not limited to, initiating a purchase of an objectrepresented in the image, sharing the image, saving the image, emailingthe image, obtaining additional information about the image and/or theobject represented in the image, etc. In some implementations, uponselection of an action, the client device may provide an indication ofthe selected action to the remote computing resource and the remotecomputing resource may initiate the selected action.

A determination is then made as to whether one of the presented actionsis selected, as in 1314. If it is determined that an action has beenselected, the selected action is initiated, as in 1316. If it isdetermined that an action has not been selected, the example process1300 completes, as in 1318. A user may select one or more of the actionsand the corresponding action is initiated with respect to the selectedadditional image. For example, if the user selects the buy or purchaseaction, the action of purchasing the object represented in theadditional image, or purchase of the additional image, is initiated.Initiation of the action may include obtaining additional detailsregarding the represented object, providing price information, sellerinformation, shipping details, etc. As will be appreciated, any varietyof actions may be associated with one or more of the additional imagesand those actions may be initiated with respect to a selected additionalimage.

FIG. 14 illustrates an example client device 1400 that can be used inaccordance with various implementations described herein. In thisexample, the client device 1400 includes a display 1402 and optionallyat least one input component, such as a camera, on a same side of thedevice as the display 1402. The client device 1400 may also include anaudio transducer, such as a speaker, and optionally a microphone.Generally, the client device 1400 may have any form of input/outputcomponents that allow a user to interact with the client device 1400,such as a mechanical control 1408. Various other input components forenabling user interaction with the device may include a touch-baseddisplay 1402 (e.g., resistive, capacitive), camera (for gesturetracking, etc.), microphone, global positioning system (GPS), compass orany combination thereof. One or more of these input components may beincluded on a device or otherwise in communication with the device.Other input components and combinations of input components can be usedas well within the scope of the various implementations as should beapparent in light of the teachings and suggestions contained herein.

In order to provide the various functionality described herein, FIG. 15illustrates an example set of basic components 1500 of a client device1400, such as the client device 1400 described with respect to FIG. 14and discussed herein. In this example, the device includes at least onecentral processor 1502 for executing instructions that can be stored inat least one memory device or element 1504. As would be apparent to oneof ordinary skill in the art, the device can include many types ofmemory, data storage or computer-readable storage media, such as a firstdata storage for program instruction for execution by the processor1502.

The memory 1504 may be a non-transitory computer readable storage mediumconfigured to store executable instructions accessible by theprocessor(s) 1502. In various implementations, the non-transitorycomputer readable storage medium may be implemented using any suitablememory technology, such as static random access memory (SRAM),synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or anyother type of volatile or permanent memory. In the illustratedimplementation, program instructions and data implementing desiredfunctions, such as those described herein, are shown stored within thenon-transitory computer readable storage medium. In otherimplementations, program instructions may be received, sent, or storedupon different types of computer-accessible media, such asnon-transitory media, or on similar media separate from thenon-transitory computer readable storage medium. Generally speaking, anon-transitory, computer readable storage medium may include storagemedia or memory media such as magnetic or optical media, e.g., disk orCD/DVD-ROM. Program instructions and data stored via a non-transitorycomputer readable medium may be transmitted by transmission media orsignals such as electrical, electromagnetic, or digital signals, whichmay be conveyed via a communication medium, such as a network and/or awireless link, such as may be implemented via a network interface.

Removable storage memory may also be available for sharing informationwith other devices, etc. The device typically will include some type ofdisplay 1506, such as a touch-based display, electronic ink (e-ink),organic light emitting diode (OLED), or liquid crystal display (LCD).The device in some implementations may include at least one imagecapture element 1508, such as one or more cameras that are able to imageobjects in the vicinity of the device. An image capture element caninclude, or be based at least in part upon, any appropriate technology,such as a CCD or CMOS image capture element having a determinedresolution, focal range, viewable area, and capture rate. The device caninclude at least one dynamic search control component 1510 for invokingthe dynamic search control, receiving adjustments to the position, size,and/or shape of the dynamic search control, obtaining an image of thedisplayed information, and/or segmenting the image. The device also caninclude at least one location component 1512, such as GPS, NFC locationtracking or Wi-Fi location monitoring.

The example client device may also include at least one additional inputdevice able to receive conventional input from a user. This conventionalinput can include, for example, a push button, touch pad, touch-baseddisplay, wheel, joystick, keyboard, mouse, trackball, keypad or anyother such device or element whereby a user can input a command to thedevice. These I/O devices could be connected by a wireless, infrared,Bluetooth, or other link as well in some implementations. In someimplementations, however, such a device might not include any buttons atall and might be controlled only through touch (e.g., touch-baseddisplay), audio (e.g., spoken) commands, or a combination thereof.

FIG. 16 is a pictorial diagram of an illustrative implementation of aserver system 1600, such as a remote computing resource, that may beused with one or more of the implementations described herein. Theserver system 1600 may include a processor 1601, such as one or moreredundant processors, a video display adapter 1602, a disk drive 1604,an input/output interface 1606, a network interface 1608, and a memory1612. The processor 1601, the video display adapter 1602, the disk drive1604, the input/output interface 1606, the network interface 1608, andthe memory 1612 may be communicatively coupled to each other by acommunication bus 1610.

The video display adapter 1602 provides display signals to a localdisplay (not shown in FIG. 16) permitting an operator of the serversystem 1600 to monitor and configure operation of the server system1600. The input/output interface 1606 likewise communicates withexternal input/output devices not shown in FIG. 16, such as a mouse,keyboard, scanner, or other input and output devices that can beoperated by an operator of the server system 1600. The network interface1608 includes hardware, software, or any combination thereof, tocommunicate with other computing devices. For example, the networkinterface 1608 may be configured to provide communications between theserver system 1600 and other computing devices, such as the clientdevice 1400 (FIG. 14).

The memory 1612 may be a non-transitory computer readable storage mediumconfigured to store executable instructions accessible by the processor1601. In various implementations, the non-transitory computer readablestorage medium may be implemented using any suitable memory technology,such as SRAM, SDRAM, nonvolatile/Flash-type memory, or any other type ofvolatile or permanent memory. In the illustrated implementation, programinstructions and data implementing desired functions, such as thosedescribed herein, are shown stored within the non-transitory computerreadable storage medium. In other implementations, program instructionsmay be received, sent, or stored upon different types ofcomputer-accessible media, such as non-transitory media, or on similarmedia separate from the non-transitory computer readable storage medium.Generally speaking, a non-transitory, computer readable storage mediummay include storage media or memory media such as magnetic or opticalmedia, e.g., disk or CD/DVD-ROM. Program instructions and data storedvia a non-transitory computer readable medium may be transmitted bytransmission media or signals such as electrical, electromagnetic, ordigital signals, which may be conveyed via a communication medium, suchas a network and/or a wireless link, such as may be implemented via anetwork interface.

The memory 1612 is shown storing an operating system 1614 forcontrolling the operation of the server system 1600. A binaryinput/output system (BIOS) 1616 for controlling the low-level operationof the server system 1600 is also stored in the memory 1612. The memory1612 additionally stores program code and data for providing networkservices that allow client devices 1400 and external sources to exchangeinformation and data files with the server system 1600. Accordingly, thememory 1612 may store a browser application 1618. The browserapplication 1618 comprises computer executable instructions, that, whenexecuted by the processor 1601, generate or otherwise obtainconfigurable markup documents such as Web pages. The browser application1618 communicates with a data store manager application 1620 tofacilitate data exchange and mapping between the data store 1603, clientdevices, such as the client device 1400, external sources, etc.

As used herein, the term “data store” refers to any device orcombination of devices capable of storing, accessing and retrievingdata, which may include any combination and number of data servers,databases, data storage devices and data storage media, in any standard,distributed or clustered environment. The server system 1600 can includeany appropriate hardware and software for integrating with the datastore 1603 as needed to execute aspects of one or more applications forthe client device 1400, the external sources and/or the search service1605. The server system 1600 provides access control services incooperation with the data store 1603 and is able to generate contentsuch as matching search results, images containing visually similarobjects, indexes of images having visually similar objects, keywords,and the like.

The data store 1603 can include several separate data tables, databasesor other data storage mechanisms and media for storing data relating toa particular aspect. For example, the data store 1603 illustratedincludes digital items (e.g., images) and corresponding metadata (e.g.,indexes, keywords) about those images. Search history, user preferences,profiles and other information may likewise be stored in the data store.

It should be understood that there can be many other aspects that may bestored in the data store 1603, which can be stored in any of the abovelisted mechanisms as appropriate or in additional mechanisms of any ofthe data store. The data store 1603 may be operable, through logicassociated therewith, to receive instructions from the server system1600 and obtain, update or otherwise process data in response thereto.

The memory 1612 may also include the search service 1605. The searchservice 1605 may be executable by the processor 1601 to implement one ormore of the functions of the server system 1600. In one implementation,the search service 1605 may represent instructions embodied in one ormore software programs stored in the memory 1612. In anotherimplementation, the search service 1605 can represent hardware, softwareinstructions, or a combination thereof. The search service, as discussedabove, is configured to receive image segments from client devices,process those image segments to determine an object represented in theimage segment, determine additional images that include representationsof visually similar objects, and/or determine keywords corresponding tothe additional images. Likewise, the search service 1605 may determine aranking for the additional images, such as an additional image rankinglist, based on the similarity of each additional image or the objectrepresented therein to the image segment and/or the object representedin the image segment. Likewise, the search service may rank thedetermined keywords based on a frequency with which each keyword isassociated with the additional images.

The server system 1600, in one implementation, is a distributedenvironment utilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 16. Thus, the depiction in FIG. 16 should be takenas being illustrative in nature and not limiting to the scope of thedisclosure.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as exemplary forms ofimplementing the claims.

What is claimed is:
 1. A computer-implemented method, comprising: undercontrol of one or more computing systems configured with executableinstructions, receiving a search invocation; presenting a dynamic searchcontrol that is to be visually positioned on a displayed information;obtaining an image of the displayed information and storing the image ina memory; determining position information corresponding to a positionof the dynamic search control; generating an image segment from thestored image corresponding to the position information of the dynamicsearch control; processing the image segment to determine a first objectrepresented in the image segment; determining a plurality of additionalimages, each of the plurality of additional images including arepresentation of an additional object that is visually similar to thefirst object; and presenting at least a portion of the plurality ofadditional images.
 2. The computer-implemented method of claim 1,further comprising: determining a plurality of keywords, each of theplurality of keywords associated with at least one of the plurality ofadditional images; and presenting, concurrently with the at least aportion of the plurality of additional images, at least a portion of theplurality of keywords.
 3. The computer-implemented method of claim 2,further comprising: prior to presenting the at least a portion of theplurality of additional images, ranking the plurality of additionalimages based at least in part on a similarity with the image segmentsuch that the presented at least a portion of the plurality ofadditional images correspond to highest ranked images; and prior topresenting the at least a portion of the plurality of keywords, rankingthe plurality of keywords based on a frequency of each keyword'sassociation with the plurality of additional images such that thepresented at least a portion of the plurality of keywords correspond tokeywords having the highest frequencies.
 4. The computer-implementedmethod of claim 3, further comprising: receiving a selection of akeyword; and re-ranking the plurality of additional images based atleast in part on the keyword; and presenting a re-ranked portion of theplurality of additional images, the re-ranked portion of the pluralityof additional images having an association with the keyword.
 5. Thecomputer-implemented method of claim 1, further comprising: receiving aselection of an image from the presented at least a portion of theplurality of additional images; presenting a plurality of actions thatmay be initiated with respect to the image; receiving a selection of anaction of the plurality of actions; and causing an initiation of theaction.
 6. A non-transitory computer-readable storage medium storinginstructions that, when executed by a processor, cause the processor toat least: visually present over displayed information a dynamic searchcontrol such that: the dynamic search control is visuallydistinguishable with respect to the displayed information, the dynamicsearch control encompasses less than all of the displayed information,and the dynamic search control may be positioned on any portion of thedisplayed information; determine a position of the dynamic searchcontrol with respect to the displayed information; determine an imagesegment of the displayed information corresponding to a portion of thedisplayed information encompassed by the dynamic search control; processthe image segment to determine a first object represented in the imagesegment; determine a plurality of additional images, each of theplurality of additional images including a representation of anadditional object that is visually similar to the first object; andpresent at least a portion of the plurality of additional images.
 7. Thenon-transitory computer-readable storage medium of claim 6, wherein anobject is determined to be visually similar based at least in part on ashape, a size, a color, or a brand of the object.
 8. The non-transitorycomputer-readable storage medium of claim 6, wherein the instructionswhen executed further cause the processor to at least: determine, basedat least in part on the plurality of additional images, a plurality ofkeywords, each of the plurality of keywords associated with at least oneof the plurality of additional images; and present, concurrently withthe at least a portion of the plurality of additional images, at least aportion of the plurality of keywords.
 9. The non-transitorycomputer-readable storage medium of claim 6, wherein: the dynamic searchcontrol is substantially circular in shape; and a radius of thesubstantially circular shape is adjustable.
 10. The non-transitorycomputer-readable storage medium of claim 6, wherein: the dynamic searchcontrol is visually distinguishable with respect to the displayedinformation by at least partially obfuscating the displayed informationthat is not encompassed within the dynamic search control.
 11. Thenon-transitory computer-readable storage medium of claim 6, wherein theposition of the dynamic search control includes a horizontal positionwith respect to the displayed information, a vertical position withrespect to the displayed information, a height, and a width.
 12. Thenon-transitory computer-readable storage medium of claim 6, wherein theinstructions when executed by the processor further cause the processorto at least: obtain an image of the displayed information; and store theimage in a memory.
 13. The non-transitory computer-readable storagemedium of claim 12, wherein: the image segment is obtained from thestored image of the displayed information; the image segment correspondsto the position of the dynamic search control with respect to thedisplayed information as represented in the stored image.
 14. Thenon-transitory computer-readable storage medium of claim 13, wherein:the image segment is less than the entire stored image.
 15. A computingsystem, comprising: a processor; and a memory coupled to the processorand storing program instructions that when executed by the processorcause the processor to at least: receive a request to present a dynamicsearch control; in response to receiving the request, visually present adynamic search control concurrently with displayed information such thatthe dynamic search control is visually distinguishable from thedisplayed information and encompasses less than all of the displayedinformation; receive at least one of a position input or a size inputcorresponding to the dynamic search control; adjust the visualpresentation of the dynamic search control according to the received atleast one of the position input or the size input; obtain and store animage of the displayed information; segment from the stored image aportion of the stored image corresponding to a position and a size ofthe dynamic search control after the visual presentation of the dynamicsearch control is adjusted; send to a remote computing resource thesegment from the stored image; receive from the remote computingresource, a plurality of ranked additional images that are visuallysimilar to an object represented in the segment from the stored image;and present at least some of the plurality of ranked additional images.16. The computing system of claim 15, wherein the program instructionsthat when executed by the processor further cause the processor to atleast: receive from the remote computing resource, a plurality ofkeywords corresponding to the plurality of additional images; andpresent at least a portion of the plurality of keywords.
 17. Thecomputing system of claim 16, wherein the program instructions that whenexecuted by the processor further cause the processor to at least:receive a selection of a keyword from the presented at least a portionof the plurality of keywords; send the selected keyword to the remotecomputing resource; receive a second ranking of the plurality ofadditional images; and present at least some of the plurality ofadditional images according to the second ranking.
 18. The computingsystem of claim 15, wherein the program instructions that when executedby the processor further cause the processor to at least: receive aselection of an image of the presented at least some of the plurality ofadditional images; present a plurality of actions that may be initiatedwith respect to the selected image; receive a selection of an actionfrom the plurality of actions; and cause the action to be initiated. 19.The computing system of claim 18, wherein the plurality of actionsinclude at least two of: an action to share the image, an action topurchase an object represented in the image, an action to store theimage, or an action to send the image.
 20. The computing system of claim15, wherein the displayed information may be any visual information thatis presented to a user, including: a web page, an email, a video, animage, or a presentation generated by a computing program.